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ABSTRACT 


The  observation  model  i/t  =  Pd/n)  +  $i,  l§ign,  is  con- 


sidered,  where  the  c’s  are  i.i.d.  mean  zero  and  variance  a2  and  /3  is 
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an  unknown  smooth  functipn.,  A  Gaussian  prior  distribution  is  i 
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specified  by  assuming  p  is  the  solution  of  a  high  order  stochastic 

V.  £-  . 

differential  equation.  The  estimation  error  6  =  p-p  is  analyzed, 
where  p  is  the  posterior  expectation  of  /?.  Asymptotic  posterior  and 
sampling  distributional  approximations  are  given  for  ||  «5||2  when  ||*[f 

'  t>!n  vtJ  $e  '  \  -'i  ■■  .  <  '■ 
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is  one  of  a  family  of  norms  natural  to  the  problem.  It  is  shown  that 


the  frequentist  coverage  probability  of  a  variety  of  (1  -  a)  posterior 

/  -  c  « 

probability  regions  tends  to  be  larger  than  1  -  a,  but  will  be 


infinitely  often  less  than  any  e  >0  as  n  *i»°°  with  prior  probability  1. 


A  related  continuous  time  signal  estimation  problem  is  also  stu¬ 
died.  - ^ 
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1.  INTRODUCTION 


In  this  article  we  consider  Bayesian  inference  for  a  class  of  non- 


parametric  regression  models.  Suppose  we  observe 


Yrd  =  P(tm)  +  Ci  1  ZiZn 


where  =i/n,  y3 :  [0, 1]  -*•  IR  is  an  unknown  smooth  function,  and 


ei*e2»‘  *  *  are  i.i-d..  random  errors  with  mean  0  and  known  variance 


a2  <  The  eni  are  modeled  as  ^(O.a2).  A  Gaussian  prior  for  0  will  now 


be  specified.  Let  m  !  2  euid  for  some  constants  a0,  .  .  .  ,  with  a,*  *  0 


L  =  £  *0* 
<=o 


be  a  constant  coefficient  linear  differential  operator  (0  =  d/dt).  Let  u 


denote  a  standard  Gaussian  white  noise  on  [0,1],  which  is  formally  the 


derivative  of  a  standard  Brownian  motion  (W(t)  :  0 §  £  £  1^.  Let  E  be  an 


m  x  m  positive  definite  matrix  and  let  5  j ....  ,  0m  be  boundary  value 


operators  of  order  m-1  or  less,  i.e.,  0*(0)  a  linear  form  in 


0(0) ,  0(1)  ,0 0(0)  ,00(1) . 0m_10(O) , 0m_10(  1) . 


Assume  that  Lg  a 0  and  (y )  =  0,  imply  y  =  0.  Then  0  is 


assumed  to  be  the  solution  of  the  stochastic  differential  equation 


L0  =  u 

B(0)~AT(O,E) 


where  B(0)  =  {B  t  (0),  .  .  .  ,  0m(0)).  Alternatively,  one  may  express  0 


through  a  simple  stochastic  integral 


■’.n'N'Ov 


m 


mm 
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l 

m  =  /  G{t.s)dW{s)  +  fix{t) 
o 

where  G  is  the  Green’s  function  for  the  operator  L  with  homogeneous 
boundary  conditions  B^fi)  =  0,  1  Si  (see  sections  3.3  and  3.4  of  Nai- 
mark,  1967)  and  fii(t)  is  a  solution  to  the  homogeneous  differential  equa¬ 
tion  {L(3i  =  0)  with  boundary  conditions  that  agree  with  fi  (5t  (fit)  =  B^fi), 
l%i  <m). 

This  specifies  the  prior  for  fi  The  Bayesian  model  thus  defined  is 
very  similar  to  others  that  have  appeared  in  the  literature  (Kimeldorf  and 
Wahba,  1970a,  1970b;  Wahba,  1978;  Wecker  and  Ansley,  1983).  More 
abstract  models  are  discussed  in  section  2. 

One  of  the  attractive  features  of  the  Bayesian  approach  is  that  in 
principle  one  can  solve  virtually  any  statistical  decision  or  inference 
problem.  In  particular,  one  can  provide  an  accuracy  assessment  for 
fin  =  E[fi\Yn]  using  posterior  probability  regions.  For  instance,  letting 
||*||  denote  £2[0,1]  norm,  one  can  in  principle  determine  a  number  A* 
such  that 

P[\\$n-fi\\2$  An|rn]  =  0.95, 

thus  giving  a  95%  posterior  probability  bound  on  the  Z,2-norm  of  the  esti¬ 
mation  error.  A  useful  large  sample  approximation  for  A*  follows  from 
results  given  below,  namely 

A*  ~  Mr*  +  1  •  645  Tn 


fWIWWllHi  1*  UJUJW 
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where 

Fn  ~  n'_1aT^1/m5(^rri .  1  -  ^m)(cr2/n)1' 1/2m 

Tn  ~  aT^1/mfl ( J£m  12-%m)(c72/n)2-  1/2m 

andB(x.y)  is  the  beta  function.  (See  (3.18),  Theorem  3.1(a),  (3.3),  (3.1), 
and  the  remarks  after  Theorem  4.1.) 

Non-Bavesians  often  find  such  Bayesian  procedures  attractive 
because  as  n  -» »,  the  frequentist  coverage  probability  of  the  Bayesian 
region  tends  to  the  posterior  coverage  probability  in  “typical”  cases.  It 
was  my  hope  that  this  would  hold  in  the  nonparametric  regression  set¬ 
ting,  thus  providing  a  methodology  for  constructing  large  sample 
confidence  regions.  Unfortunately,  the  hoped  for  result  is  false  in  about 
the  worst  possible  way,  viz. 

liminf  P[i|0n-0|j2£  AJ0]  =  0.  a.s.  (1.3) 

fl 

Thus,  if  one  fixes  a  sample  path  from  the  Gaussian  prior,  then  the  fre¬ 
quentist  coverage  probability  of  the  region  f/S:  ||/J-$n||2  ^  A*}  will 
infinitely  often  be  arbitrarily  small  asn-»»,  for  almost  all  sample  paths. 

Nonetheless,  for  fixed  n  (large),  the  frequentist  coverage  probability 
is  at  least  95%  for  “most"  sample  paths  in  the  following  sense: 

lim  F[P[|!ftt-/?|]2S  |/3]  §  0.95]  =  1-a 

n  ■*« 

where  1-a  depends  on  m  as  indicated  in  the  following  table: 


m 

2 

3 

— 

4 

10 

ao 

1  -a 

89.4% 

90.5% 

91.2% 

92.7% 

95% 

(see  Corollary  3.4). 

It  will  be  indicated  in  section  5  that  the  “typical”  settings  wherein 
the  frequentist  coverage  probability  converges  to  the  Bayesian  posterior 
coverage  probability  depend  critically  on  finite  dimensionality  of  the 
parameter  space.  One  would  naturally  conjecture  that  a  result  like  (1.3) 
holds  whenever  the  prior  is  infinite  dimensional. 

The  procedure  analyzed  here  differs  from  that  advocated  by  Wahba 
(1983)  and  Wecker  and  Ansley  (1983)  in  the  following  ways. 

(1)  These  authors  use  a  “mixed  effects”  model  wherein  (3  =  + 

and  Px  is  given  a  proper  prior  of  the  type  above  but  /30  is  unprior- 
ized  and  lies  in  a  finite  dimensional  space  (or  can  be  said  to  have 
an  improper  Lebesgue  prior).  The  resulting  regions  are  a  mix¬ 
ture  of  confidence  and  posterior  probability.  I  believe  (1.3)  will 
hold  for  this  setting  as  well. 

(2)  These  authors  only  look  at  pointwise  confidence  intervals  for  (3(t ) 
for  some  t  €  [0,1]  rather  than  the  global  confidence  regions 
treated  here.  I  have  not  succeeded  at  analyzing  the  pointwise 
intervals,  but  I  believe  that  (1.3)  will  hold  for  them  as  well.  See 
section  5  for  some  relevant  discussion. 
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(3)  We  have  assumed  a 2  is  known,  but  I  am  sure  that  this  is  inconse¬ 
quential. 

(4)  These  authors  assume  /S  =  6/?o  where  /So  has  a  given  prior  and 
6  >0  is  an  unknown  scale  factor  which  is  estimated.  I  do  not 
know  if  our  negative  results  still  hold  in  this  setting.  In  particu¬ 
lar  (1.3)  depends  on  the  law  of  the  iterated  logarithm  fluctua¬ 
tions  of  the  bias  E[^n  -f}  |  /S]  about  its  mean  (Lemma  3.2).  Such 
fluctuations  undoubtedly  impact  the  smoothing  parameter  esti¬ 
mation  procedure  of  Wahba  (1983),  known  as  generalized  cross 
validation.  (See  also  Craven  and  Wahba  (1979)  and  Speckman 
(1983).)  Thus,  an  empirical  Bayes  approach  may  avoid  this  prob¬ 
lem. 

(5)  Wahba  (1983)  considers  the  true  function  (i  to  be  fixed  and 
smoother  than  one  generated  by  the  Gaussian  prior.  I  believe 
(1.3)  may  hold  for  many  such  smooth  functions. 


Section  2  presents  an  analysis  of  an  abstract  version  of  the  Bayesian 
linear  model.  In  section  3,  a  continuous  time  analog  of  (1.1)  is  investi¬ 
gated.  In  section  4,  it  is  shown  that  the  discrete  model  (1.1)  can  be  well 
approximated  by  the  continuous  time  model.  Section  5  contains  some 
concluding  remarks. 


... 


&  < 


2.  THE  ABSTRACT  GAUSS-BAYES  LINEAR  MODEL 


In  this  section,  we  formulate  and  solve  a  general  version  of  the 
abstract  linear  model  when  the  error  vector  is  modeled  as  Gaussian  with 
known  covariance  and  a  Gaussian  prior  is  used  for  the  parameter  vector. 
Well  known  results  about  Gaussian  measures  on  Banach  spaces  will  be 
used,  for  which  Kuo  (1975)  is  an  excellent  reference;  see  also  Kuelbs 
(1970,  1971),  and  Kallianpur  (1971).  We  will  drop  the  subscript  n 
throughout  this  section  as  we  only  consider  a  fixed  prior  and  observation. 

Suppose  c  is  (modeled  as)  a  mean  0  Gaussian  random  vector  on  a  real 
separable  Banach  space  Y,  and  Y  is  the  generating  Hilbert  space  (GHS) 
for  the  Gaussian  measure  L(e)  =  the  distribution  of  £  on  Y.  (Y  is  also 
known  as  the  reproducing  kernel  Hilbert  space  for  L{e);  see  Kuelbs 
(1970),  Kallianpur  (1971),  or  Kuo  (1975).  Let  /3  be  a  mean  zero  Gaussian 
random  vector  on  a  real  separable  Banach  space  0  with  GHS  0,  and 
assume  /?  and  e  are  independent.  Now  let 

X  :  Q-*Y 

be  a  bounded  linear  operator,  called  the  design  operator,  and  suppose  we 
observe 

Y  =  JT/3  +  oe  , 

where  o>0  is  known.  We  wish  to  estimate  /?  from  Y.  To  do  this,  we  will 
show  that  a  posterior  distribution  exists  and  characterize  it. 


As  a  remark,  we  note  that  Y  plays  little  role  in  what  follows  -  it  is  Y 
that  is  important.  If  it  were  not  for  the  technical  problem  that  L(e)  will 
not  live  on  Y  when  dim  Y  =  we  could  carry  out  the  analysis  using  only  Y 
with  no  mention  of  Y.  Kallianpur  and  Karandikar  (1985)  have  an  elegant 
approach  to  problems  in  this  vein.  Our  approach  here  is  in  the  more 
classical  style  using  ordinary  measure  theory,  for  which  it  is  necessary  to 
keep  Y  around. 

Proposition  2.1.  There  is  a  regular  conditional  posterior  distribution 
L{{i\Y)  ,  and  in  fact  L((3\Y)  is  absolutely  continuous  w.r.t.  the  prior 
Lift)  ,  written  Z,  (/?  |  Y)  «  L  (fi) . 

Proof.  The  assumption  that  the  range  R  (X  )  C  Y  guarantees  the  sam¬ 
pling  distribution  L{Y  |/3)  «  L(at)  by  Theorem  3.1,  p.  118  of  Kuo  (1972). 
An  elementary  Fubini  argument  completes  the  proof,  a 

As  the  inclusion  map  t:  0  -*  0  is  continuous,  it  follows  that  the  restric¬ 
tion  X  of  X  to  0  is  in  B  (0,  Y),  the  set  of  all  bounded  linear  operators  from 
0  -*Y.  Let  X*  e  B  (Y,  0)  denote  the  adjoint  and  put 

U  =  X*X  e  B  (0) , 

where  5(0)  =  5(0,0).  The  description  of  the  posterior  is  most  easily 
accomplished  using  the  singular  value  decomposition  of  X,  given  next. 
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Lemma  2.2.  There  exist  complete  orthonormal  systems  (abbreviated 


c.o.n.s .)  v=  1,2,  ■  ■  ■  y  for  0  and  <7jn  :  i/=  1,2,  •  •  -  ^  for  R{X) , 
the  closure  of  R(X)  in  Y,  and  constants  <^a^:  u=  1,2,  •  •  •  ^  suc/i  that 
UTpv-  a%i/v  for  V'  v ,  and 

Xij/V  =  avT)v  ,  V  v  (2.1) 

X*rjv  =  avipv  ,  V  v  (2.2) 

Ecc*  <  oo  .  (2.3) 


Proof.  X  =  X  o  t  and  t  is  a  compact  operator.  Hence  X ,  U  and  V  =  XX* 
are  compact.  U  is  also  self  adjoint  and  nonnegative  definite  on  0,  so 
there  are  nonnegative  eigenvalues  af.otf,  •  •  •  and  eigenvectors  '  '  ' 

such  that  tyjy  is  a  c.o.n.s.  for  0,  and  UTj/v=  afyu  (Theorem  1.8,  page  8  of 
Kuo,  1975).  We  assume  w.l.o.g.  that  a f  ^  af  ^  •  •  •  .  Put  tjv  =  a^lXif/v  for 
oi jy  >  0  and  then  (2.1)  and  (2.2)  hold.  Now  we  have  p  =  with 

convergence  L{p)  a.s.  in  0,  so  Xp  =  2 <*v{P,‘'l'v}  Vv-  As  this  latter  series 


converges  L((3)  a.s.  in  Y,  and  <(p,ip . (p^^y^ .  •  •  •  are  i.i.d.  N( 0,1),  it  fol¬ 
lows  that  Ea®  <  ».  Alternatively,  one  checks  that  V  is  the  covariance  of 
Xp  in  Y,  so  trace  V  -  Ea*  <  °°.  □ 


Even  though  p  e  9  with  probability  0,  the  “stochastic  linear  func¬ 


tional"  p  -» is  defined  a.s.  as  a  measurable  map  0-»IR,  for  each 


fixed  0e0.  The  properties  of  these  maps  are  given  by  Kuelbs  (1971)  and 
Kuo  (1975),  Lemma  4.7,  p.  78.  Similar  remarks  hold  for  c  and  , 

**  «  IV 

rjeY.  Note  that  (Y  =  (Xp,r]y^-¥  a^c.py^,  a.s.  Since  X(3eY,  the 
term  ^Xp.rjy^  is  defined  as  an  ordinary  linear  function.  We  also  have 
Karhunen-Loeve  expansions:  if  ej.e^  •••  is  any  c.o.n.s.  for  0,  then 

M  ___ 

fi  =  e„  with  convergence  in  0,  almost  surely.  See  Kuelbs  (1971). 

v  x  8 

Lemma  2.3.  Let  YV~^Y, and  ,  v-  1,2,  •  •  •  .  Then 

Lifia*"-  in  =  Yu)  =  ^^((a2  +  a,2r1a,ri/,[l  +  (aI^a)2r1) 

ie.,  the  posterior  distribution  of  Pifa,  •  ••  is  that  of  independent  nor¬ 
mal  random  variables  with  the  indicated  means  and  variances. 

Proof.  Put  tv  =  (z, Tjv^y  Then  Pi, fa,  ....  £i,£2,  ■  •  •  are  i.i.d.  N( 0,1).  Also 

Y„  =  a.jpv+  aty.  The  result  follows  from  this  and  elementary  calcula¬ 
tions.  □ 

As  the  inclusion  t:  0-*0  is  continuous,  every  (£0*,  the  dual  of  0. 
defines  by  restriction  an  element  of  0*.  If  we  identify  0  =  0*,  then 
i*  €  B  (0*,0)  satisfies  ,  V  0e0.  An  inner  product 

type  notation  is  being  used  for  the  (0*,©)  duality  pairing. 


Theorem  2.4(a).  The  posterior  L{0  Y)  on  0  is  Gaussian  with  mean 

0  =  (a2/  rUy'X'Y  .  Z.(/3.£)  as  f2  4) 

and  covariance 

V  =  i(/  +•  a~zU  )~li* 

(b).  <5  =  0-0  is  independent  of  Y  ,  and  has  a  Gaussian  distribution  on 

0  with  me  an  0  and  covariance  V. 

Remark.  For  (2.4)  to  make  sense,  we  need  to  show  X*Y  is  defined  a.s.  and 
an  element  of  0.  Put 

X*Y  =  £YUX*  Vv  =  Z<xuYui,u. 

1/  U 

Note  that  ZE[{avYjf]  =  Ea^a^  +  o2)  <  =»,  so  the  series  on  the  right  con¬ 
verges  a.s.  in  0.  Furthermore,  we  have  for  V  0  that 

(e.xny^  =  (x  s.r).~ 

as  one  can  check  from  series  expansions  of  both  sides. 

Proof.  0  =  Z,0vipv  with  a.s.  convergence  in  0.  Hence,  if  f  €  0*. 

«.eyf'W  =  w*(<.‘f„ys.g  =  Eft-e  <•*>,. 

.  2  ? 

As  ZV<(1* (,if/vy  =  it  follows  from  Lemma  2.3  that 

L(<(t,0y^  ^  \  Y)  is  Gaussian  with  mean 
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!L  =  ^v(az  +  a^)-1avYl/<\*^,Tp  ^ 

and  variance 

t2  =  £„[1  + 

Note  that 

(<■,  t^/ +[/r1x*y)_e  =  ((azI+U)~h^,X*Y'}9 

1/  '  ® 

Also  <^,  l(I +azU)~iL*^y^  -  t2.  This  completes  the  proof  of  (a). 

*s« 

Put  <5„  =  ^d.V'A  =  (iv-  (<r2  +  a^)_1al/rv.  It  is  easy  to  check  that  <5„  is 
Gaussian  and  independent  of  Yv,  hence  Y .  The  rest  of  (b)  is  easy.  □ 


Now  we  introduce  a  parametrized  family  of  Hilbert  spaces  which  is 
natural  for  the  problem.  For  convenience,  assume 

R{X*)  is  dense  in  0  (2.5) 

where  R{X*)  =  R{U)  is  the  range  of  A'*.  Also,  put 

Yv  ~  &V  ^  •  Vv  ~  ®l/  • 


,*  I, 


r~ 


* 

i 


P 
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=  Lyl(e,M^ya(e2M^\. 

We  collect  some  elementary  facts  about  these  spaces. 

Proposition  2.5.  Suppose  (2.5)  holds,  then 

(a)  ^0,^  =  (Q’U us  1. 2.  •  •  ■  ^  is  a  c.o.rus.  /or  8o  . 

(b)  0,  =  0  - 

(c)  extends  to  a  Hilbert  space  isomorphism  from  80  to  the  Y 
closure  of  R  (X)  . 

(d)  If  p>r  then  8p  C  0T  with  continuous  inclusion. 

( e )  Letting  tp  denote  the  inclusion  8  -*  Bp  where  p  S  1 ,  and  iden¬ 
tifying  0 p*  with  0p  in  the  usual  way.  we  have  ifi*  =  Ux~p  . 

(f)  0p  supports  L(f!)  if  and  only  if  Yi7C~X  <  00  • 

Proof,  (a)  and  (b)  are  elementary.  For  (c)  note  that  X<pv  =  tjv  and 
/Q.Utp^r  -  /X  .  For  (d)  note  that  ||0||p  £  (max'y£”T)||0||r.  and  the 

maxvyQTT  is  achieved  at  a  finite,  positive  value  by  (2.3).  For  (e),  note  that 
Ux~p<pv  =  7u{1~p)<Pu  and  by  comparison  of  the  series  defining  the  two  sides 
of  (ipd.iiy  =  (d,ip*T)y  ,  one  obtains  that  ip*r\  =  £ <Pv}  <Pv  which 

gives  the  result. 


Turning  to  (f),  suppose  £  y£-1  <  «>.  Then  the  operator  D  :  0-*0  given 

U 

by  D  9  =  is  Hilbert-Schmidt,  so  the  norm  |||  •|||  on  0 

given  by  ||id|||  =  \\D  0||j  is  a  measurable  norm  (Exercise  17,  page  59  of 
Kuo,  1975)  and  hence  the  completion  of  0  under  111*111  supports  L(/3) 
(Theorem  4.1,  page  63  of  Kuo,  1975).  One  easily  checks  that  |||  *|||  =  ||*|!p. 
Conversely,  suppose  0p  supports  L(/3),  and  let  us  calculate  the  covariance 
operator  V  for  L(/3)  on  0p.  For  0£0, 

C*™),  =  E(e.f>y‘  =  «[( Z7g(o,r„\(/i.ruyo)‘ 

V  0  V  1 

=  £7£<0.^>o<frp_10.^>o  =  (e.uf’-'e}' . 

and  since  0  is  dense  in  0p,  we  can  identify  V  -  Up~x  =  tptp*.  Now  V  must 
be  a  trace  class  operator  (Theorem  2.3(a),  page  29,  and  Definition  2.2, 
page  16  of  Kuo,  1975),  and  the  eigenvalues  of  V  are  \y8~x\,  so  Ey*-1  <  a 
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a.  CONTINUOUS  TIME  ESTIMATION 

In  this  section  we  consider  a  continuous  time  analog  of  the  problem 
from  section  1.  It  will  be  seen  in  the  next  section  that  the  sequence  of 
experiments  with  discrete  observations  can  be  sufficiently  well  approxi¬ 
mated  by  a  sequence  of  experiments  with  continuous  observations  that 
the  limit  theory  of  the  latter  is  inherited  by  the  former. 

Suppose  t  £  1^.  is  the  stochastic  process  of  (1.2)  and  let 

(e(0: 0£<  £1).  be  a  standard  Gaussian  white  noise.  Consider  the 
sequence  of  experiments  with  observations 

rn(t)  =  ?{t)  +  ,  OStSl  . 

where  er  >  0  is  a  constant.  We  write 

=  <*n  *  O2/*-.  (3.1) 

Now  the  GHS  for  L(e)  is 

Y  =  i2[0,l]. 

0,  the  GHS  for  L  (/?).  is  equal  as  a  set  to  the  Sobolev  space 

W?  =  »T[0,1]  -  \f '■  f  maps  [0,1]-»1R,  and 

/ , Df . Dm~xf  are  absolutely  continuous 

with  Dmf  eZ-2(0.1]|  • 

The  0  inner  product  is 

l 

=  b (eyz-'Btt)  +  j£(l  e){L()  . 

One  can  show  that  ||*j|e  is  equivalent  to  where 
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\mlf  =  \\Dmd\\iz  +  n®iift- 

Now  let  0  be  a  Banach  space  which  will  support  L  (/?)  and  is  continu¬ 
ously  imbedded  in  Z,2  (i.e.,  9  cia  and  has  a  stronger  norm),  e.g., 
0  =  c"1-1  or  .  Let  X  :  0  -»Z,g  be  the  imbedding  operator  (X  9=  9),  so 
Y  =  Xfi  +  anE.  Then  X ,  the  restriction  of  X  to  0.  is  the  imbedding  of  0  in 
L  2-  To  identify  X* ,  start  with 

l  i 

f  (l  9)(lx*t})  +  Bieyz-'Bix*!))  =  f  &n 
o  o 

valid  for  9eQ  and  t)  eL  Apply  integration  by  parts  to  the  integral  (as  in 
section  1.5  of  Naimark,  1967)  and  express  the  l.h.s.  as 

fiL'LXX^e  +  ^ZiCjoiX^DieiO)  +  Cj\{X*rj)D^  9(1)] . 

0  ;=0 

Here  L *  is  the  formal  adjoint  of  L .  Also,  C;i(£)  is  a  boundary  value  opera¬ 
tor  of  order  2m  -j  whose  highest  order  term  is  of  the  form 
±a£D2m~* f(i).  Thus  f  =  X* 7}  can  be  obtained  as  a  solution  of  the 
differential  equation 


L*L<;  =  Tj 


CjiiO  =  0  ,  OS;  <m  ,  i  =  1,2. 


To  derive  =  E[f3\Yn]  -  (A*/  +  U)~xX*Yn,  it  is  convenient  to  use  the  fol¬ 


lowing  characterization:  /5n  is  the  unique  /?£  0  such  that  for  all  0, 


*$w>\  =  . 


& 

« 
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If  one  writes  this  out  and  uses  integration  by  parts,  there  results  that  /?n 
is  the  solution  of  the  stochastic  differential  equation 

(x nL*L  +  1)?  =  rn 

Cji0)  =  0  ,  0  Zj<m,  1  =  1,2. 

The  <pv's  and  yu's  are  the  eigenvectors  and  eigenvalues  of  the  differential 
operator  L*L  with  boundary  conditions  C*  =  0,  0 <m,  i  =  1,2.  Since 
the  highest  order  term  of  (f)  is  ±a^Z}2m“^(i),  the  boundary  condi¬ 
tions  are  regular  by  the  argument  on  pp.  60-61  of  Naimark  (1967).  By 
Theorem  2,  pp.  64-65  of  Naimark, 

7  v  =  aT£(m')2m(l  +  0(u~3/z))  ,  u-»».  (3.2) 

(Note  that  the  eigenvalues  have  to  be  real  and  positive.)  Put 

C*(A.p)  =  S^(1+xri/)"fc  • 


These  quantities  will  have  many  uses.  In  particular  note  that 

*  kc^k-p) 

EWt-Etfmt  »  KC*(K.p)- 


Using  (3.2),  one  can  show  that  if  -  J£tti  <  p  <  1  -  fam,  then  as  A  -» 0 


Ck(X.p)  ~  /x2mp[i+zam]-fcdz 

*o 

=  1/2771,  k-p-  l/2m)  (3.3) 


where 
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«  =  rr~1aT^1/m 

and  B(u,v)  =  r,(u)r(u)/r(u+'u)  is  the  beta  function.  The  derivation  for 
the  asymptotic  formula  for  Cfc(A,p)  follows  as  in  Corollary  5.4  of  Speck- 
man  (1984).  The  evaluation  of  the  integral  is  given  in  Gradshteyn  and 
Ryzhik  (1965),  formula  3.194.2,  page  285. 

Now  0O  =  Lz  and  ©i  3  ,  where  3  means  equal  as  sets  and  with 

equivalent  norms.  From  this  it  follows  that 

0P  a  .  OSpSl.  (3.4) 

The  proof  is  most  easily  accomplished  with  the  K -method  of  interpola¬ 
tion;  see  Theorem  3.4  of  Cox  (1986). 

One  of  the  reasons  the  norms  ||»(|p  are  so  useful  is  because  we  can 
derive  explicit  limiting  distribution  results  as  A*  -»  0  for  the  norm  of  the 
error  vector  <5n  =  /3  -  /3n . 

Theorem  3.1.  Let  -  1/2771  <  p  <  1  -  l/2m  . 

(a)  Put 

Pmifl)  =  KCliK'P) 

r|(p)  =  2A2c2(An,2p). 

Then 

i([l|4»ll,2-M.l/T„)  =>  Af(O.l) 


OS  71  “>ao  . 
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(b)  Put 

n„G °.fl  -  ^StT'O+WJ-W-O 

1/ 

Vnip)  =  2X^[C3(\n  ,2p)  +•  XnC4(Xn ,  1  +2p)] . 

Then 

kWn\\p-^-^nVrtn\^)  =>  #(0.1) 
as  n  -» »  ,  L(/3)  as. 

Remarks.  In  view  of  Theorem  2.4(b),  part  (a)  gives  the  asymptotic  poste¬ 
rior  distribution  for  ||<5J||.  Part  (b)  gives  the  asymptotic  sampling  distri¬ 
bution  of  ||<Jn||p.  Note  that  fl*  is  random  as  it  depends  on  0. 

Proof.  For  (a),  note  that 

iim!  = 

V  U  V  9 

and  (6n  ,  <6n  •  •  •  are  independent  normal  with  mean  0  and  vari¬ 

ance  Var =  Xn7i/(1  +  \»7v)-1-  The  result  follows  from  Lindeberg’s 
central  limit  theorem  after  suitable  truncation  of  the  infinite  series  for 

II l<M?- 

Part  (b)  is  somewhat  more  difficult.  The  conditional  distribution  of 
given  /3  is  that  of  independent  normals  with  mean 

*n7i/A/(l  +  \i7i/)_1  and  variance  +  ^7v)-2-  From  this  it  follows  that 
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k 


is 


¥ 


K 


E 


B 


£[|I<UI*I0]  =  Ain  +  fin  and 


Var[||<5n||||/S]  =  rjn  +  4X^  £  *  - l)(l  +  \n7v)-*. 


From  (3.3)  we  have  rjn  2-3p-i/2m  whiie  from  Lemma  3.2  below,  the 
second  term  above  is  j)  (X^-?p-1/''2m),  a.s.  Now  the  proof  follows  as  in  part 
(a)  □ 


Lemma  3.2.  Let  r>  1/2 vn  and  k  be  such  that 


k  >  t  +  1/4  m 


(3.5) 


Put 


i/=l 


Then  for  some  C  e  (0,°°)  , 

limA*1/4"1  (loglogn)"^cj(n)  =  C  ,  a.s. 


n- 


limAn1/4m  (loglog7i)”^ej(n)  =  -C  ,  a.s. 


Proof.  The  first  step  is  to  show  that  there  is  a  probability  space  carrying 
a  probabilistic  replica  of  u(n),  also  denoted  cj(n),  and  a  standard  Wiener 
process  \  W(t ):  t  £0}  such  that  the  process  given  by 


V(s)  =  -/ W(st)g(t)dt  ,  s^O 


g{t)  =  (d/dt)h(tZm)  ,  h(z)  =  zT/(\+x)k 
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satisfies 


|fi ;(n)  -  2 ^((cXn)-1/2Ta)  =  o  fa  1/4m(loglogn)^  (3.6) 


almost  surely,  where 


_  _  -  2  _2m 

C  =  OmTT 


Note  that  (3.5)  guarantees  convergence  of  the  integral  defining  V. 
Let  S  v  -  £/=1 2-%/  -  1)  and 

Dx{n)  =  2"^a(n)  =  -2‘S’y[/i(Xn7l/4.,)-/i(\,7j] 


where  the  last  formula  follows  by  partial  summation.  By  Theorem  2.6.1, 
page  107  of  Csorgo  and  R6v6sz  (1981),  there  is  a  probability  space  carry¬ 
ing  a  version  of  (S',,:  i/=  1,2,  •  •  •  }  and  a  Wiener  process  (^(f):  t  ^0}  such 
that  for  all  <5  > 0,  asi/-»« 


\SV-W(y)\  =  jo(t^)  a.s. 


(3.7) 


Let 


Dz (n)  =  “2  ^W[/l(An7v+l)  " 


Note  that 


I  h(*7v+i)  "  I  =  *fov+i  -  yv]  I  AW  I 


where  7„  ^  7„  s  yv+l.  Utilizing  (3.2)  one  ceui  show  that 

7v+i  ~  7v  =  c277u/an'1(l  +  0(^^)). 


(3.8) 


Also, 


|/i'(X7„)|  *  C,,[(Ai^n)T-1(l  +  cAi/2mr1:  +  (Ai^ni+cAi^"1)-^*1)] 


ms. 


for  some  C \  £  (0,°°).  Collecting  things  together  we  have 


\D  \(n)  -  D%{n)\  g  CzK^H 

1/ 

[(Xni^m)T-1(l+cXnuan)-fc  +(Xni^m)T(l+cXnt^m)-(fc+1)]Xn1/2m  (3.9) 

where  C2  £  (0,°°)  is  random  (from  (3.7))  and  depends  on  6.  Note  that  the 
summation  above  tends  to  a  finite  integral  asn-*».  Hence,  taking  6  < 
gives  that 

|Z?i(n)-Z?2(n)|  =  £(V1/4m(loglogn)^)  . 


Next  we  show  that  D2(n)  is  suitably  close  to 
fl3(n)  = 

1/ 

Two  applications  of  the  mean  value  theorem  along  with  (3.8)  yield 
\D2(n)~  D2(n)\  % 

V 


for  some  C  €  (0,°°),  almost  surely.  If  one  uses  the  fact  that  \W(v)\  =  0  (u) 
a.s.  and  applies  the  same  argument  that  was  used  on  (3.9),  it  can  be 
shown  that  the  last  expression  is  0(1)  a.s.,  which  is£(A^1/4m(loglogn)^). 
(The  assumption  t  >  1/2 to  is  needed  here  to  guarantee  that  a  summa¬ 
tion  converges  to  a  finite  integral.) 


Now  let  Z?4(n)  =  V({c  A*)-  1/2m  ).  Then 

ID,(n)-Dt(n)l  S  K)-W*EZA, 


(3.10) 


where 


zv  =  sup  iino-wi 

is  a  sequence  of  i.i.d  random  variables  and 

dyiv  =  !y'(l>Xn]1/2m  [y  +  «ni/])l 

for  some  anu  £  [0,1].  Now  P[ZU>  y]  =  AP[W{\)^y],  so  as  u -»<*>, 

Zv  -  0( \ogv)  a.s. 

Plugging  this  into  (3.10)  and  using  a  familiar  argument  shows  that  for  any 
d  >  0 

|Z?3(n)-f)4(n)|  *  0(\^s)  a.s. 

as  n  This  completes  the  proof  of  (3.6). 

Now  we  show  that  if  sn  -  ,  then  for  some  C  e  (0,°°) 

lim  (sn  log  log  sn  )~%V (sn )  =  C  a.s.  (3.11) 

n 

Since  loglogsn  =  log  log  (n~la~1/mn1//^m  )  -  loglogn,  the  lemma  will  fol¬ 
low  from  (3.6)  and  (3.11).  (Note  that  the  corresponding  lim  in  (3.11)  will 
be  -C  by  symmetry.)  Put 

Y(u)  =  e~^iV(eu)  ,  -  °°  <  u  <  «> 

Uji  =  logs*  =  A  +  (2m)"1  log  n  ,  n  =  1,2,  •  •  • 

where  A  =  -logrr(71/Tn  is  a  constant.  One  can  check  that  Y{u)  is  a  sta¬ 
tionary  mean  zero  Gaussian  process  with  covariance 

m 

T(u)  =  e"^u  / G(e~ut)G(t)dt 

o 
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G{t)  =  h{tm). 

Equation  (3. 1 1)  will  follow  by  showing 

lim  (logi^ =  2^r(0)  a.s.  (312) 

n 

To  prove  this  latter  we  will  apply  Case  2  of  Theorem  D  of  Qualls  (1977) 
with  a  =  2.  Assuming  this  result  applies,  it  states  that  for  c  >0, 

/3[>,(^)>  r(0)c0°gx*n)^]  =  0  or  1 

according  as  the  integral 

o* 

f  f~c*/2(iog  t  )c*/4(log  t  -  )£log  log  t  )~%dt  <  °°  or  =  »  . 

This  latter  integral  converges  for  c  >  2^  and  diverges  for  c  <  2^,  which 
establishes  (3.12). 

In  order  to  apply  the  aforementioned  result  in  Qualls  (1977),  it  is 
necessary  to  check  that  Y  has  continuous  paths,  T(u)  =  o.(  1/log u)  as 
u  and  F'(0)  <  0.  To  show  Y  is  a.s.  continuous,  it  suffices  to  show  that 
V  is  a.s.  continuous  on  [0,°°),  and  for  this  it  suffices  to  show  V  is  a.s.  con¬ 
tinuous  on  any  finite  interval  [0,6]  and  then  let  b  -*  °°  through  some 
countable  set.  Now  let  A  be  an  event  of  probability  1  on  which  IV  is  con¬ 
tinuous  and  for  which  there  exists  T o  (depending  on  the  path  of  W)  such 
that  for  all  T  ^  T0 

(2T  log  log  T)~%  sup  |  fK(f+s)- W(t)\  z  2.  (3.13) 

OStSf/2 

osssr/2 

Such  an  event  A  exists  by  equation  (1.2.4),  page  30  of  Csbrgo  and  R6v6sz 


(1981).  Let  c  >  0  be  given.  Pick  T j  ^  T0  such  that 


4  f  4(6f  log  log  btfe  g  (0 :  dt  <  c/2  . 

Ti/Zb 


(3.14) 


This  is  possible  by  (3.5).  Now 


suP<fc  \V(Sl)-V(s2)\  <  f  sup  W(stt)-W(s2t)\  g{t)  dt 


0 Ss,5s2S6 


T^b 

+  f  sup  \W(s1t)-W{s2t)l  g{t)\dt  =  /,+/2. 

Q  OMjiSgSt) 


Since 


sup  I^(sj0- ^(^201  £  slip  |  W{u+v)~  W{u)\ 

0Ssii52S6  OSuSbt 

0  ivSbt 


it  follows  from  (3.13)  and  (3.14)  that  / 1  <  t/2.  A  straightforward  uniform 
continuity  argument  shows  that  there  is  a  <5  >  0  such  that  jsj-s2j  S  <5 
implies  I2  <  c/2.  This  completes  the  proof  of  continuity. 


To  show  F(u)  =£(l/logu)  as  u  -» we  have  for  u  >0  and  r 


T(u)  $  e-(%+T/Zm)u 


f  tt/m[\+TX/2m]~kdt 


+  f  f(2T-fc)/2m  dt  +  gfcu/Sm.  y  t(r-k)/mdt 


=  0  (e  ~(!4+T/2m)u)  +  o  (g((T-*)/^T7»  +  ^)u) 


As  all  the  exponents  are  negative,  the  desired  result  follows.  If 
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t  =  y^k-m,  then  the  middle  integral  is  treated  differently  but  the  result 
still  holds. 

To  show  that  P'(0)  <  0,  first  note  that 

oo  eo 

P'(0)  =  34 / Gz(t)dt  +  / [Ztg(t)  +  tzg\t)]G(t)dt  .  (3.15) 

*o  o 

The  calculation  is  carried  out  by  differentiating  under  the  integral  sign 
which  is  justified  by  the  dominated  convergence  theorem  and  by  the  fact 
that  there  exists  a  constant  c  such  that  for  all  u  e  [-1,1],  |G(e~uf)|, 

|  tg{e~ut )| ,  and  t2g'(e~ut)\  are  all  bounded  by  c  min fl,£  j2m(T_fc),  which 
is  square  integrable  on  (0,°°)  by  (3.5).  Noting  that  the  quantity  in  brack¬ 
ets  on  the  r.h.s.  of  (3.15)  is  d/dt[tzg(t )],  an  integration  by  parts  will 
yield 

OO  00 

P'(0)  =  &/ Gz(t)dt  -  f  tzgz{t)dt  . 

*0  0 

Let  1 1  denote  the  first  integral  in  this  last  expression  and  /2  the  second. 
Then  integration  by  parts  gives 

oo 

/j  =  -2/ tg(t)C{t)dt 

and  Cauchy-Schwarz  applied  to  this  integral  gives  I \  <21^-1  ^  which 
implies  I x  <  4 /2  and  hence  P'(0)  =  ^ /4  —  <  0.  □ 

Corollary  3.3.  Let  -  1/2 m  <  p  <  1  -  l/2m  .  Suppose  that  A*  >  0  is 


chosen  so  that 
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■^[HAt  ~  P\\p  —  I  ^»]  =  l-« 


(3.16) 


for  some  a  e  (0,  l)  .  Then 

lim  inf  P[||^n -^!||  ^  ^  |^]  =  0  ,  a.s. 


n-*— 


(3.17) 


Proof.  By  Theorem  3.1(a) 

~  +  2«Tn  (3.18) 

where  za  is  the  upper  100a%  point  of  the  #(0,1)  distribution.  By 
Theorem  3.1(b) 

|/,[||^n-^25An]/3]-$((2aTn-nn)/7?n)|  -  0 

a.s.  as  n  -*»,  where  $  denotes  the  #(0,1)  distribution  function.  (Note 

that  convergence  in  distribution  to  #(0,1)  implies  uniform  convergence 

I 

i  of  the  distribution  function  to  $.)  Now  Tn/7jn  tends  to  a  finite  limit  and 

i 

i 

Vn  - 

by  (3.3).  Hence,  by  Lemma  3.2 

t 

|  lim^inf  (zarn  -  0n)/r)n  =  -« 

J  which  proves  (3.16)  □ 

) 

l 

I 

! 


Corollary  3.4.  With,  the  setup  of  the  previous  corollary,  the  prior  proba¬ 
bility  that  the  (1-a)  Bayesian  posterior  probability  region 
\Q€.%:  AnJ  is  conservative  in  its  freqventist  coverage 
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probability  tends  to  $(za(l -r^)(l -r2)-^)  where 

r2  =  (1  -p  -  l/4m)(2p/3  -  l/Grm)  . 


Proof.  We  wish  to  calculate 

p\fi- 

The  difference  between  this  and 

^[^((2«Tn -Qn)/r)n)  *  1-a]  =  ^[On  ^  (Tn  -’In)*#] 

tends  to  0.  One  can  show  as  in  Theorem  3.1(a)  that 
^(^n/(Tn  -*7n)^)  =>  ^(0,1).  Using  this  and  (3.3)  one  can  show  that  the 
last  displayed  probability  tends  to  a  limit  of  the  form  $(z„i?)  where  R  is 
an  algebraic  form  in  some  beta  functions.  If  one  substitutes 
B  (x ,y)  =  r(x)r(y)/T(x +y)  and  uses  the  recurrence  relation 
r(x  +  l)  =  xr(x),  then  the  claimed  result  follows  after  some  algebraic 
manipulation,  o 


Wf  z*  f* K*  £9  V.^ ’ li n  *  u  *  iTO  JHT*V  ■niX'i.^ii  «m  «r j 
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4.  DISCRFTE  TIME  ESTIMATION 

In  this  section  we  consider  the  original  discrete  time  problem  in 
(1.1),  where  /3  is  given  the  prior  specified  in  (1.2).  By  virtue  of  the  next 
result,  it  follows  that  Theorem  3. 1  and  Corollaries  3.3  and  3.4  hold  in  this 
setting  as  well,  thus  justifying  the  statements  made  in  the  introduction. 

Theorem 4.1.  Suppose  that  m>2  and 

-1/2  m  <  p  <  1  -  1/2  to 

and  that 

J?  |  e*  | 2+a  <  » 

where 

6  *  6/(2ro-3). 

Then  there  exists  a  probability  space  carrying  versioTis  of  (J , 
ti.tz,’ •  •  and  a  sequence  {?n(<  ):  0£f  £  lj  of  Gaussian  white  noises 
such  that  if 

Yni  =  ft(i/n)  +  Ei  ,  l£i£n 

ft.  =  •  «»-?»-? 

?»«)  ■  +  .  OifSl 

a.  »«[?!?„].  a. -a.-* 

then 
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Remarks.  Note  that  ||?n||p  =  Op(\l-p“ 1//Zm)  follows  from  £,||3’n|||  =  /^(p). 
Since  ||<5n -3„ll|  *  it  follows  that  ||<5n|||  =  Op(X^~p' l/Zm). 

Hence 

!ll<5nll|-!l?«l!p2|  *  ll<5n-?nllp[ll<5„||pH-t|dnllp] 

Now  rn  «  A^~p_  i'/4m  and  jjn  »  X^~p~1/4m,  so  we  may  replace  the  continu¬ 
ous  time  estimation  error  (now  denoted  <?n)  with  the  discrete  time  esti¬ 
mation  error  <5n  in  Theorem  3.1,  and  the  results  still  hold.  Corollaries  3.3 
and  3.4  depend  only  on  these  results  and  the  properties  of  which 
depends  only  on  0. 


m 


Proof.  We  have 


K-JUU*  i  2||rn-Sn||!  +  2||<iJn-3n)fl|! 


where 


=?„-£[?»!  P]  .  Sn  =  3»-£[?>»IP] 


B„{>  =  E$n-P\l S]  = 


Ke  =  •£■[&, -pm  =  [(Ki+ur'u-iip. 


We  will  show  that  each  of  the  two  quantities  on  the  r.h.s.  of  the  inequality 


above  has  the  correct  order  asn->«. 


rn  =  (X nl  +  Uj-'Xn'lj 


*.  \.v V  .\  %■  ■ 


WI"UUW  .T  MJ  -j  J^T VV>-V.PVJJ-y?-J-7? 


-30- 

and  following  the  notation  of  Cox  (1984b)  put 

Fn  =  (XnZ+f/r1^. 

Now  we  apply  some  results  in  section  4  of  Cox  (1986).  One  can  show  that 
for  all  xx,xz  in  0 

|((f/-£/n)x1(x2)>8l  £  CVi”l[||x1||1/m||x2||0  +  ||Xllloll*2lll/m] 

where  C  €  (0,®)  is  a  constant,  so  Assumption  4.1(f)  of  Cox  (1986)  holds 
with  kn  ~  Cn~x,  s  =  1/ra,  j  =  2.  See  the  remarks  after  this  Assumption 
for  a  proof  of  the  above  inequality.  Now  by  equation  (4.6)  of  Cox  (1986), 

£||r»-f;il!  S  Cn-!![C2(Xn.p  +  l/m)£'||T-nliJ  +  C2(X.l.p)£'||rn||?/m]. 

This  latter  inequality  can  be  inverted  for  p  =  0,  1/m  to  give  bounds  on 
£’||rn|||  and  i,||rn||f/m  in  terms  of  £||t£||$  and  E\\rn When  these  are 
substituted  back  into  this  inequality  and  the  estimates  on  C2  are  used 
there  results 

£||r„-r;il|  S  Cn-3v-S''-». 

It  is  easily  seen  that  this  last  quantity  is  js  (\^-p)  as  m  £  2. 

Now  put 

l 

Tn  =  on'*/  (KI  +  Ur'tWdBnoFM 
o 

where  Fn(t)  -  n~1[nt ]  as  in  (3.7)  of  Cox  (1984),  then  following  (3.9)  of 
that  paper, 

=  n_1C2(An,p  +  l/7n)o(n"d/(24'd)) 


a.s.  Using  the  estimate  on  C2,  the  definition  of  Xn,  and  the  hypothesis 
that  5  >  6 /(2m -3),  it  follows  that  the  last  displayed  quantity  is  .o 
a.s. 

Next,  we  have  from  (3.10)  of  Cox  (1984b)  that 

Ifo-Snllp  =  Op(n"1(n'1logn)C2(Xn,p-(- 1/m)) 

and  this  is _Op(A4~p).  This  completes  the  proof  that  j|rn  -sn|||  =  Op(\£~p). 

Finally  we  must  take  care  of  |j(5n  -Bn)P\\£  where  Bn  and  8n  denote 
the  bias  operators  for  the  respective  problems.  Let 
max  (p,  1/m  {  <  r  <  1  -  1/2 m,  and  let  |j*||TiP  denote  the  operator  norm  of 
a  linear  operator  from  0T  to  0p.  Then  by  equation  (4.6)  of  Cox  (1986), 

lia.-5.llr,  s  Cn~,(C3^(Xn,p+-  l/mjlj^nlko  + 

CAK.P  *  1  /m)Kl/Zm  Il5.llr.v-m  + 

CAK.p)K-'/‘m  I|5» llr.O  ♦  ^(^llft.lkl/ml  ■ 

Now  by  Theorem  2.3(c)  of  Cox  (1986),  \\Bn\\r^  £  CX^r~p^  as  long  as 
p$T$p+  2.  Plugging  this  into  the  above  expression  and  using  the  esti¬ 
mates  on  C2  yields 

HA..  *5.llr,  *  C  j*>-P-V»»*r,H  . 

Now 

As  t  <  1  -  1/2 m,  ||0j||  <  «  a.s.,  by  (3.2)  and  Proposition  2.5(f),  and  so  the 
last  expression  is  a.s. 
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5.  CONCLUDING  REMARKS 

To  provide  more  insight  into  the  foregoing  analysis,  consider  the  case 
where  dim©  =  k  <  Then  it  is  easy  to  obtain  the  analog  of  Theorem  3.1. 
Calculating  the  asymptotic  posterior  distribution  of  |jdn|||,  we  have 

£(II<U|>  =  k  i,KyS^+K7v)-'*i 

ll/=l 

Since  (1  +\,7V)-1  —  lasn-*“  for  v  =  1,3,  .  .  .  ,  k,  it  follows  that 

i(Vll«»llD  ->  k  t  yt'S 

I/*  1 

where  the  distribution  on  the  right  is  a  weighted  sum  of  x2  random  vari¬ 
ables.  The  corresponding  calculations  for  the  sampling  distribution  are 

i(ll'5„ll|lfl  =  II,J  7S(l  +  K7.r*‘S 

1/=1 

+  7>(it»,7,)-!tA + x{  e  i +K7.r2e; 

i/*i 

The  second  and  third  summations  on  the  r.h.s.  (which  result  from  the 
bias)  are  each  0p(X,7l),  so 

i<V'IIM?l0)  =>  k  iylrt 

v=i 

Thus,  the  asymptotic  posterior  and  sampling  distributions  are  identical, 
so  a  (1-a)  posterior  probability  region  of  the  form  in  (3.16)  has  (asymp¬ 
totically)  the  right  coverage  probability  from  the  sampling  point  of  view. 
This  results  from  the  fact  that  the  random  (conditional  on  p)  variability 
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dominates  both  the  sampling  and  posterior  distributions,  with  the  bias 
E  \fin  -  p  1 0]  being  of  smaller  order. 

In  the  case  when  dim©  =  the  bias  and  random  variability  are 
always  of  the  same  order  of  magnitude.  This  is  the  driving  force  behind 
Corollaries  3.3  and  3.4. 

Concerning  extensions  of  the  major  results  (Theorem  3. 1,  Lemma  3.2, 
and  Theorem  4. 1),  one  would  like  to  look  at  more  general  Gaussian  priors 
and  more  general  quantities  them  ||dn||p-  I  believe  that  analogous  results 
can  be  obtained  (for  ||<5n|lp  hi  any  setting  wherein  7„«  vr  for  some  r  >0 
(see  Cox  (1986)).  The  specific  setting  here  edlowed  us  to  use  the  sharp 
eigenvalue  asymptotics  (3.3)  at  some  points  in  Lemma  3.1  (see  (3.8)).  The 
extension  to  other  quadratic  forms  can  also  probably  be  accomplished. 
As  in  Wahba  (1983),  we  would  like  to  construct  a  confidence  interval  for 
/?(£ )  for  some  t  €  [0,  l].  In  the  setup  of  section  3,  note  that 

i(MO)  =  »(oaE(i^7,)-WW) 

V 

£(M  OIW  =  Z(i  +  *.7r)"W(0) 

where 

Mn  =  K  £  7^(  1  +  K7v)~lPi0v(t )  • 

1/ 

I  conjecture  that  both  variances  above  are  «  1/2m,  and  also  that 

lim  sup  [An(1-1/Zm)(loglog  l/Xn)]~^Afn  e  (0,«)  ,  a.s. 
n  -*«• 

This  would  imply  analogs  of  Corollaries  3.3  and  3.4  for  the  ‘‘confidence’’ 
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